Moisture sensor and windshield fog detector

ABSTRACT

A control system for automatically detecting moisture on the windshield of a vehicle. The automatic moisture detecting system includes an optical system for imaging a portion of the windshield on to an image array sensor, such as a CMOS active pixel sensor. The voltages of each of the pixels which represents the illumination level is converted to a corresponding gray scale value by an analog digital converter. The gray scale values corresponding to the image are stored in memory. The spatial frequency composition of the gray scale values are analyzed to determine the amount of rain present. In order to provide a control signal to control the operation of the windshield wipers of the vehicle as a function of the amount of moisture present. The system is also adapted to detect the level of fog both on the interior of the windshield as well as the exterior of the windshield. By providing a system for automatically detecting the presence of fog on the interior and exterior of the windshield, serious performance limitations of known automatic rain sensors are eliminated.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.09/347,093, now U.S. Pat. No. 6,097,024, entitled “MOISTURE SENSOR ANDWINDSHIELD FOG DETECTOR,” filed on Jul. 2, 1999, by J. Stam et al.,which is a continuation U.S. application Ser. No. 08/931,118, filed Sep.16, 1997, now U.S. Pat. No. 5,923,027. The entire disclosures of whichare incorporated herein by reference.

TECHNICAL FIELD

The present invention relates to a system for automatically detectingthe presence of moisture on a surface such as the surface of awindshield of a vehicle, in order to automatically actuate the vehicle'swindshield wipers and/or defroster or defogging system.

BACKGROUND ART

In conventional windshield wiper systems, the windshield wipers areactuated based upon the elapsed time between wipes rather than themoisture level on the exterior of the windshield. During conditions ofrelatively consistent rainfall, for example, the time interval can beadjusted to correspond to the amount of time in which the rainaccumulates to the point of the desired visibility level. Unfortunatelythe rate of rainfall may vary dramatically over a given period of time.Additionally, traffic conditions may also cause varying amounts of rainto fall on the windshields due to traffic conditions, such as a passingtruck or the like. As a result, during such conditions, the wiper timeinterval must frequently be adjusted which can be cumbersome.

Various systems are known which automatically control the intervalbetween wipes of the windshield wipers based upon moisture on thevehicle windshield. In some known systems, various coatings are appliedto the vehicle windshield. Electrical measurement of those coatings areused to provide an indication of the moisture content on the windshield.Unfortunately such methods require relatively expensive processes whichmakes such system commercially non-viable. Other systems forautomatically sensing the moisture content on a vehicle windshield arealso known. For example, optical systems are known which measure thedifference of reflected light of a dry windshield versus a wetwindshield. Unfortunately, the optical method is susceptible tointerference from external light sources and thus provides inadequateperformance. Other known systems must be adhered to the windshield whichcomplicates the windshield replacement. As a result of suchcomplications, moisture sensors are rarely found on vehicles.

Another system for automatically detecting the moisture content on awindshield is disclosed in Japanese Laid Open Patent Application No.Hei>(1995)-286130 which describes the use of a charge coupled device(CCD) image sensor to image a portion of the vehicle windshield in orderto detect raindrops. The system described therein computes the sum ofthe differences between each pixel and the average of all pixels.Unfortunately headlamps of oncoming vehicles will create a bright spotin the image which would be difficult to completely blur and likely beinterpreted as rain. Moreover, in order for such a system to workeffectively, images from the distant scene must be completely blurred.Otherwise, there will be dark and light regions in the distant scene.Although there is no optical system disclosed in the Japanese laid openPatent Application for accomplishing this objective, it would be verydifficult to develop an optical system to completely blur an oncomingheadlamp. Failure to blur oncoming headlamps could cause falsetriggering of the system disclosed in the above-identified Japanese laidopen Patent Application.

Another problem with automatic rain detection systems is the inabilityof the system to detect the operation of the windshield wipers. Incertain cold climate conditions, the windshield wipers are known tofreeze to the windshield. In such a situation, since the moisture is notbeing removed by the wipers, an automatic rain sensing device wouldcontinuously command the wipers to actuate, even though the wipes arefrozen to the windshield, potentially damaging the windshield wipersystem.

Another known problem with known systems is the inability to detect fogon the interior and exterior of the windshields. As mentioned above,automatic moisture detection systems, such as disclosed in theabove-identified laid open Japanese Patent Application, are based uponthe ability to detect raindrops on the windshield. When a uniform fog ormist covers the vehicle windshield, systems, such as the systemdisclosed in the Laid Open Japanese Patent Application, are unable tosense such moisture on the exterior of the windshield. As a result,during such a condition, the windshield wipers will have to be manuallyactuated, thereby partially defeating the purpose of an automatic rainsensor and windshield wiper control system making the feature a lot lessdesirable.

In other situations, fog develops on the inside of the windshieldindependent of the moisture content on the exterior of the windshield.In such a condition automatic rain sensing systems, such as disclosed inthe Laid Open Japanese Patent Application, are unable to detect themoisture content on the exterior of the vehicle windshield until afterthe fog on the interior of the windshield is cleared. In such acondition, a defroster or defogger system would have to be manuallyactuated to remove the interior fog on the windshield. The automaticrain sensor would not be operable during such a condition until the fogon the interior of the windshield is sufficiently cleared.

The desirability of having automatic rain sensing is to have a systemwhich automatically controls the windshield wipers during typicalclimatic conditions, such as rain, snow and fog. When the wiper systemhas to be operated manually during such typical conditions such afeature becomes undesirable.

DISCLOSURE OF INVENTION

It is an object of the present invention to provide a system whichsolves the problems of the prior art.

It is yet another object of the present invention to provide a systemfor automatically detecting the moisture content on the windshield of avehicle.

It is yet another object of the present invention to provide a systemfor automatically detecting moisture on the windshield of a vehicleduring common climatic conditions, such as rain, snow and fog.

It is yet a further object of the present invention to provide a systemfor automatically detecting the presence of fog on the exterior of awindshield.

It is yet another object of the present invention to provide a systemfor automatically detecting the presence of fog on the interior of awindshield.

It is yet a further object of the present invention to provide a systemfor automatically sensing the windshield wipers crossing a portion ofthe windshield.

Briefly the present invention relates to a system for automaticallydetecting moisture on the windshield of a vehicle. The automaticmoisture detecting system includes an optical system for imaging aportion of the windshield onto an image array sensor, such as a CMOSactive pixel sensor. The voltage of each of the pixels, which representsthe illumination level, is converted to a corresponding gray scale valueby an analog to digital converter. The gray scale values correspondingto the image are stored in memory. The spatial frequency composition ofthe gray scale values are analyzed to determine the amount of rainpresent in order to provide a control signal to control the operation ofthe windshield wipers of the vehicle as a function of the amount ofmoisture present. The system is also adapted to detect fog both on theinterior of the windshield as well as the exterior of the windshield. Byproviding a system for automatically detecting the presence of fog onthe interior and exterior of the windshield, serious performancelimitations of known automatic rain sensors during typical climaticconditions are eliminated.

BRIEF DESCRIPTION OF DRAWINGS

These and other objects of the present invention will be readilyunderstood with reference to the following specification and attacheddrawing wherein:

FIG. 1 is a physical diagram which illustrates a vehicle windshield andan attached rearview mirror illustrating a system in accordance with thepresent invention.

FIG. 2 is an enlarged view of a portion of the system for sensingmoisture on the exterior a windshield in accordance with the presentinvention.

FIG. 3 is a physical diagram of a system in accordance with an alternateembodiment of the for sensing fog which illustrates the projection of abeam of light onto the windshield for fog detection.

FIGS. 4a and 4 b are computer simulated spot diagrams which illustratethe performance of the optical system in accordance with the presentinvention during moisture and non-moisture conditions, respectively.

FIG. 5 is a flow diagram for the system in accordance with the presentinvention.

FIG. 6 is a block diagram of the system in accordance with the presentinvention.

BEST MODE FOR CARRYING OUT THE INVENTION

An automatic moisture sensing system in accordance with the presentinvention is able to detect moisture on the windshield of a vehicle inorder to automatically control the vehicle's windshield wiper, defrosterand/or defogging systems. The system for automatically sensing moistureon a vehicle windshield eliminates many of the performance deficienciesof known automatic moisture sensing systems at a commercially viablecost. As used herein, the term moisture is used to designate varioustypes of moisture and precipitation which can be found on the windshieldof a vehicle during various climatic conditions, such as rainfall, snowfall, ice, fog as well as other substances that are commonly depositedon a vehicle windshield such as bugs, dust and the like. The system isable to provide superior performance to other known systems duringrather common climatic conditions, such as ice, fog and varying levelsof rain and snow fall and the like.

As will be discussed in more detail below, a portion of the windshieldis imaged onto an image array sensor. An optical system which forms aportion of the present invention causes raindrops and other sources ofmoisture on the windshield to be sharply focused while distant objectsbeyond the windshield are severely blurred in the image. The processingsystem analyzes the image for sharp discontinuities caused by the edgesof the water rain droplets or other moisture and by random focusing ofthe distant objects by the droplets. These discontinuities representhigh spatial frequency components. The magnitude of high spatialfrequency components is a measure of the amount of the rain or othermoisture on the vehicle which can be used to automatically control thevehicle windshield wipers. In an alternate embodiment of the inventionthe system is adapted to sense fog on the interior and exterior of thewindshield in order to prevent spurious operation of the automaticmoisture sensing system. As such, the present invention eliminates manyof the various performance limitations of known automatic moisturesensing systems.

In yet another alternate embodiment of the present invention, the systemis able to detect operation of the windshield wipers in order to preventspurious operation and damage to the windshield wiper system duringconditions when the windshield wipers are stuck or frozen to thewindshield. As discussed above, the system analyzes the image of aportion of the windshield for sharp discontinuities which will haverelatively high spatial frequency components. The magnitude of thesehigh spatial frequency components is used to represent the measure ofmoisture or other substances on the windshield. Thus, dust, bugs andother substances that will initially be treated as moisture. However, asdiscussed above, the system has the ability to automatically sense anoperation of the windshield wiper blades. Thus, if the substance, whichmay be ice, dirt, cracks or other substances not removable by thewindshield wipers; remains on the windshield after one or more wipes,the system in accordance with the present invention may be configured toignore such substances in order to prevent further spurious operation ofthe vehicle windshield wiper system.

Referring to FIG. 1, the automatic moisture sensing system in accordancewith the present invention generally identified with the referencenumeral 20. The automatic moisture sensing system may be mountedstationary in the mounting bracket 22 of an automobile rearview mirror24 or alternatively mounted in the rear portion of the rearview mirrorhousing 24. The automatic moisture sensing system 20 includes an imagesensor mounted, for example, 2-3 inches behind the vehicle windshield 26with the optical axis substantially parallel to ground or slightlyangled to the ground. The angle of the windshield 26 in a modernpassenger car is about 27°. Such a configuration may cause the raindropsand other moisture to be at a different distance from the image sensordepending on where the moisture is with respect to the field of view ofthe image sensor. To help compensate for this problem, the image sensormay be angled approximately 10° toward the windshield 26 such that thetop of the sensor 20 is moved closer to the windshield 26.

There are four main components to the automatic moisture sensing system20: an imaging optical system; a light emitting diode; an image sensor;and a processor. The imaging optical system is shown best in FIG. 2while the image sensor is illustrated in FIG. 2 and in FIG. 6. A flowdiagram for the microcontroller is illustrated in FIG. 5.

Imaging Optical System

The imaging optical system, generally identified with the referencenumeral 30 (FIG. 2), is used to image a predetermined portion of thewindshield 26 onto an image sensor 32 such that objects at theapproximate distance of the windshield 26 are sharply in focus at theimage plane while objects at a longer distance are out of focus andblurred. The area of the windshield 26 that is imaged must be largeenough that the probability of receiving raindrops during relativelylight rain conditions is significant. Moreover, the imaged area of thewindshield must also be in the area of the windshield that is wiped bythe windshield wipers.

The imaging optical system may include a single biconvex lens 33, usedas an imaging lens. The lens 33 may have a diameter of 6 mm; a front andrear radius of curvature of 7 mm for each surface and a center thicknessof 2.5 mm. The front surface of lens 33 may be positioned 62 mm from theouter surface of the windshield 26. The imaging lens 33 may be carriedby a mechanical lens mount 34 which forms a stop 36 of about 5 mmdiameter directly in front of the lens 33. The image sensor may belocated about 8.55 mm from the rear surface of the lens 33 and asmentioned above, slightly angled by about 10°.

More elaborate optical systems, for example, with multiple elements,aspherical elements, or defractive objects could all be used especiallyif shorter distance from the windshield is a desired feature. However,since the collected images are not for photographic purposes, suchoptical quality is not necessary in an application for automaticmoisture detection. A single lens may also be used molded in acrylic orother clear plastics at a relatively low cost. Various companiesincluding Polaroid and Kodak specialize in high performance moldedplastic optics.

FIG. 4 illustrates a computer simulation of the performance of theimaging system illustrated in FIG. 2. In particular, FIG. 4a is a spotdiagram of the imaging of approximately parallel light rays from arelatively distant object on the optical axis onto an image plane. FIG.4b is a spot diagram of the imaging of a point on the optical axis atthe distance of the outer surface of the windshield. Upon comparison ofthe spot diagrams of FIGS. 4a and 4 b, it is evident that the opticalsystem is able to blur light coming from distant objects while focusinglight from objects at the windshield distance.

Occasionally when driving up a hill, the vehicle could be positioned insuch a way that the sun is directly imaged by the device. The radiativeloading caused by this alignment may damage the image sensor 32 overtime. In order to alleviate such a problem, an electrochromic filter maybe used to temporarily eliminate most of the sunlight from the imageplane. Other optical electronic or optical mechanical devices could alsobe used.

Image Sensor

The image sensor 32 may be a CMOS active pixel sensor. CMOS active pixelsensors are a recent breakthrough in imaging technology that allow lowcost, high sensitivity imaging on a chip manufactured in a CMOS process.Such CMOS active pixel sensors have several advantages over othersensors including low power consumption, popular CMOS productiontechniques, low cost and ability to integrate additional circuitry onthe same chip, variable read out windows and a variable lightintegration time. Such CMOS active pixel sensors are commerciallyavailable from Photobit LLC, La Cresenta, Calif. While CMOS active pixelsensors have substantial advantages' other image sensors are alsosuitable and are considered to be within the scope of the presentinvention. The size and number of pixels is determined to image an areaof the windshield sufficiently large and enough detail to adequatelydetect light rain while remaining cost effective. For example, a 64×64active pixel, 40 μm pixel size array will image approximately a 25 mm×40mm, on a standard passenger car windshield.

Processing and Control

A block diagram of the automatic moisture sensing circuitry is shown inFIG. 6. As mentioned above, a predetermined portion of the windshield 26is imaged onto an image array sensor 32. The analog voltage of each ofthe pixels within the sensor 32 is converted to digitized gray scalevalue by way of an analog to digital converter. The analog to digitalconverter 35 is operated under the control of a timing and controlcircuit 37 which, in turn, is controlled by a microcontroller 38. Thetiming and control circuit 37 is described in detail in U.S. patentapplication, Ser. No. 08/933,210, entitled Control Circuit For ImageArray Sensors, by Jon Bechtel and Joseph Stam, assigned to the sameassignee as the assignee of the present invention, filed on even date,hereby incorporated by reference. A suitable microcontroller 38 is aMotorola type 68HC08XL36. However, it is commonly known that suchmicrocontrollers do not contain sufficient random access memory (RAM) tostore entire image from 50×50 pixel image sensor. In such a situation, awindowing feature of the CMOS imaging sensors may be used toalternatively image and process different regions of small enough sizefor the onboard RAM of the microcontroller 38.

As discussed above, the system analyzes the digitized gray scale valuesfor sharp edges which are representative of rain drops or other moistureby analyzing the spatial high frequency components. The magnitude of thespatial high frequency components is used to control a windshield wipermotor control 40 such that the frequency of wiping of the windshieldwiper blades (i.e. time interval between wipes) is controlled as afunction of the amount of moisture on the windshield. As will bediscussed in more detail below, the system also is able to detect thefog on the interior and exterior of the windshield. Thus, themicrocontroller 38 may also be used to automatically control the vehicledefroster or defogging system 42. In order to provide selectivity of asystem, a driver on/off sensitivity control circuit 44 may be provided.This control circuit 44 may be used in special circumstances for examplewhen the vehicle is in an automatic car wash to prevent spuriousoperation of the system.

Once an image is acquired by the image array sensor 32, the luminance oneach pixel, represented by an analog voltage, is converted to a digitalgray scale value by the analog digital converter 35. These values arewritten to memory, which may be on board the microcontroller 38 andprocessed by the microcontroller 38 or alternatively a digital signalprocessor.

Rain is detected by quantifying the discontinuity resulting from sharpedges of the raindrops on the windshield. These sharp edges are causedby the focused image of the rain or other moisture droplet along withthe random optical imaging of far field objects by the droplets or othermoisture. As discussed in “Digital Image Processing” by R. C. Gonolezand R. E. Woods, Addison-Wesely 1992, hereby incorporated by reference,the images may be analyzed in terms of their spatial frequencycomposition. Spatial frequency composition analysis is analogous toFourier analysis, commonly used in both digital and analog signalprocessing. The process of taking a Fourier transform of a signal anddetermining its frequency composition can readily be applied to twodimensional signals. When the two dimensional signal is an image, it iscommon to use the term spatial frequency. The spatial frequencycomposition of an image can be evaluated using a two dimensional Fouriertransform of the image. The transform is given by equation (1) asfollows: $\begin{matrix}{{F\left( {\omega_{y},\omega_{y}} \right)} = {\int_{- \infty}^{\infty}{\int_{- \infty}^{\infty}{{f\left( {x,y} \right)}^{{- j}\quad \omega_{x}x}^{{- j}\quad \omega_{y}y}\quad {x}\quad {y}}}}} & (1)\end{matrix}$

where:

f (x,y)is the value of the pixel in the original image located at pixelx,y; F (ω_(x), ω_(y)) is the value of the Fourier transform of the imageat pixel location ω_(x), ω_(y); j is the complex number {square rootover (−1+L )}.

Equation (1) describes the Fourier transform for continuous infinite twodimensional signals. This function can be readily adapted to discrete,finite two dimensional signals resulting from digital images. Applyingspatial frequency analysis techniques, the rough edges or “roughness” ofan image can be relatively accurately quantified. For example, a Fouriertransform can be performed on a very blurry image. In such an analysisthe value of F (ω_(x), ω_(y)) for low magnitudes ω of spatialfrequencies ω_(x), ω_(y) will be high while the value of F (ω_(x),ω_(y)) at high magnitudes of ω_(x), ω_(y) will be low. The value of F(ω_(x), ω_(y)) where ω_(x), ω_(y) are both 0 is always the average grayscale value of the image.

Alternatively, a Fourier analysis of a sharply focused image with manyedges will result in the values of F (ω_(x), ω_(y)) for large magnitudesof ω_(x), ω_(y) being high. A digital filter can be used to selectparticular spatial frequency regions. A relatively simple implementationof such a filter for image processing uses a 3×3 matrix supplied to a3×3 pixel neighborhood as illustrated below:

A B C D E F G H I

A new image g(x,y) can be formed which is the resultant image ofapplying the filter to the current image. The image may be processed ina looping fashion for every pixel with a value of f(x,y) at a locationdefined by the variables x and y. In the above matrix, the location ofthe coefficient E corresponds to the current pixel at x and y. The pixelat location x and y in the new image has a value given by equation (2)below:

g(x, y)=A·f(x−1, y−1)+B·f(x, y−1)+C·f(x+1, y−1)+D·f

(x−1, y)+E·f(x,y)+F·f(x+1,y)+G·f(x−1,y+1)+H·f(x,y+1)+I·f(x+1,y+1)  (2)

A special filter commonly used is a Laplacian filter. The Laplacian isthe second-order derivative of a two dimension function f(x,y) given byequation 3: $\begin{matrix}{{\nabla^{2}f} = {\frac{\partial^{2}f}{\partial y^{2}} + \frac{\partial^{2}f}{\partial x^{2}}}} & (3)\end{matrix}$

This Laplacian function can be implemented in discrete space using the3×3 matrix described above with the coefficients as follows: E=4; B,D,F& H=−1 and the rest of the coefficients zero. Other coefficientcombinations can also be used to compute variations of the discreteLaplacian as long as the coefficient E is positive and the rest arenegative and that the sum of all coefficients is zero. The spatialfrequency response of any 3×3 filter is determined by equation (4)below: $\begin{matrix}{{H\left( {\omega_{x},\omega_{y}} \right)} = {\sum\limits_{m = {- 1}}^{1}{\sum\limits_{n = {- 1}}^{1}{{{h\left( {m,n} \right)} \cdot ^{{- j}\quad \omega_{x}m}}^{{{- j}\quad \omega},n}}}}} & (4)\end{matrix}$

where:

H(ω_(x), ω_(y)) is the frequency response of the filter for frequenciesω_(x) and ω_(y); the function h(m,n) describes the coefficients of thematrix above; the coefficient E is the value of h(0,0), A is the valueof h(−1,−1), etc; j is the complex number {square root over (−1+L )}.

By analyzing the frequency response of the discrete 3×3 Laplacian filterusing equation (4), it is evident that the discrete 3×3. Laplacian is ahigh pass filter. By modifying the coefficients, the particular responseof the filter can be adjusted. Additionally, a 5×5 or larger filter canbe used for even finer control over the response.

Rain drops and other moisture can be detected by using a 3×3 Laplacianfilter described above. Every pixel is examined individually in a loopfashion and a variable used to store the total amount of “moisture”detected. The Laplacian for each pixel is computed using the formula forg(x,y) described above with the Laplacian coefficients.

A flow diagram in accordance with the present invention is illustratedin FIG. 5. Initially in step 46′ an image of the windshield is acquired.As mentioned above, the optical system in accordance with the presentinvention images the scene in such a way that distant objects are out offocus and objects at the windshield distance are in focus. Thus, ifthere is no moisture or other matter on the windshield, only a blurryimage of distant objects will be captured. A blurry image will have arelatively low high frequency spatial component. Thus, the value of theLaplacian in such a situation will be relatively low. If there is rainor other moisture on the windshield, the drops will be in focus and theimage will contain relatively large high frequency components. Despitethe blurring of distant objects by the optical system 30, oncoming headlamps from other vehicles which are relatively bright may contribute asignificant high frequency spatial component. In order to filter thiscomponent out, pixels which have a gray scale value above the saturationlevel of the analog to digital converter (i.e. pixels with a gray scalevalue at or near 255) may be skipped.

In step 48, the Laplacian of each pixel is computed and the results arestored. More particularly, if the magnitude of a Laplacian is above agiven threshold, indicating a large enough high frequency spatialcomponent to indicate rain or other moisture, this value is summed withthe values of the other pixels to indicate the total value of rain orother moisture which is compared against a threshold, which may be auser set threshold as indicated in step 50. If the sum of the Laplacianof each of the pixels is greater than the threshold as determined instep 52′ windshield wipers are actuated in step 54. Otherwise the systemloops back to step 46 and acquires a new image of the windshield.

The threshold indicated in step 52 may be a fixed threshold or avariable threshold. In applications where variable threshold is used,the threshold may be a user set threshold implemented by a control knobor slide with a voltage output. This voltage output may then be sampledand converted to a digital value which is appropriately scaled forcomparison with the sum of the pixels which indicate the total amount ofrain.

In order to prevent spurious operation of the system, the operation ofthe wiper blade is sensed in step 56. In particular, a small sub windowof the image array sensor may be selected to allow more cycles to beprocessed per second in order to detect a relatively rapidly movingwindshield wiper. An image of the wiper is acquired in step 56. Eachimage is processed using two one dimensional high pass filters; one forthe vertical direction and one for the horizontal direction as indicatedin step 58. Since the windshield wiper will appear as a vertical line inan image, there should be a significantly higher high frequencycomponent in a horizontal direction than in a vertical direction. Thus,a vertical high pass filter may be implemented using a 3×3 matrixdescribed above with the coefficient E set to 2, coefficients B and Hset to −1 and the rest set to zero. A horizontal filter is implementedwith the coefficient E set to 2, the coefficients D and F set to −1 andthe rest set to zero. The sum of each component is tallied in the samefashion as used to compute the Laplacian discussed above. In step 60,the ratio of the horizontal component to the vertical component iscomputed. If the horizontal component is much greater than the verticalcomponent, a vertical line is assumed to be present in the imageindicating the presence of the windshield wipers.

If the wipers of the automobile are designed in such a way that thewipers are never approximately vertical when it crosses the rain sensingarea, the filters described above can be modified to accommodate such aconfiguration. For example, various other edge detection methods wellknown in the art of image processing can also be used. Additionally, ifthe wiper speed for the vehicle windshield wipers is so fast that itblurs slightly in the image for the necessary exposure time, thehorizontal filter can be modified to subtract the pixels two positionsto the left and right of the current pixel instead of the pixelsimmediately next to the current pixel.

After the wiper has cleared the image sensing area, as indicated in step62, another image of the windshield is acquired in step 64 for which theLaplacian is computed. This computation is used as a zero pointmeasurement that may be subtracted from all subsequent measurementsuntil the next wipe. In this way, long term high frequency spatialcomponents in the image of dirty windshield, cracks, scratches andfrozen ice will not contribute to the detected amount of rain.

If the windshield wiper is not detected within a given time frame, thesystem assumes that a malfunction has occurred, which can be caused as aresult of the windshield wiper being frozen to the windshield. Duringsuch a condition, the operation of the moisture sensor in accordancewith the present invention can be suspended for a period of time toallow the ice to thaw. If outside temperature information is available,freezing climate conditions can be taken into account to decide if thewipers are failing because of a mechanical malfunction or due to ice.

The system is also able to adapt to varying light levels. In particular,during selected cycles, the average gray scale value of the image may becomputed. If this value is high, indicating an over exposure to light,the image sensor integration time in the following cycle may be reducedto lower the average brightness. Similarly, if the light level is low,the integration time may be increased. In relatively dark conditions,some image sensors may not be capable of collecting enough light in areasonable time to adequately image moisture such as raindrops. In sucha situation, an additional illuminator may be provided to brieflyilluminate the area of interest from behind while the image is beingtaken. If the windshield of the vehicle is not highly absorbent toinfrared radiation, a near infrared illuminator can be used as long asthe wavelengths are within the detectable region of the image sensor. Aninfrared illuminator has the benefit of not being visible to the humaneye and thus not distracting to the driver.

Fog Detector

In order to eliminate many of the operational deficiencies of knownmoisture sensing systems, the system in accordance with an alternateembodiment of the invention includes a system for detecting fog on theinterior and the exterior of the windshield. As illustrated in FIGS. 2and 3, a light emitting diode (LED) may be used for detecting fog bothon the interior and the exterior of a vehicle windshield 26. Twodifferent embodiments of the fog detection systems are disclosed both ofwhich rely on the difference in the way the light from the LED 66 isreflected from the windshield 26 in the presence of fog. Fog detectionmay be done in alternate processing cycles with the rain detection. Inparticular, many cycles may be used for rain and moisture detectionbetween fog detection cycles due to the slow onset of windshieldfogging. At the beginning of the fog detection cycles, an image may beacquired using a window containing the expected location of the spots.

In the first embodiment, a light source that is either highly collimatedor focused to a point at the windshield distance can be used. The lightsource can be either infrared emitting or visible emitting dependingupon the absorption characteristics of the windshield. Infrared sourcesare preferable since they are not visible to the human eye and thereforedo not pose a distraction. An infrared LED 66 may be used along with alens 68 of a focal length equal to the windshield distance as generallyshown in FIG. 2. The LED 66 may be positioned a few millimeters abovethe main optical assembly and angled so that the projected light asshown in FIG. 3 is aimed at the position of the main optical axis on thewindshield 26. The LED 66 is initially turned off and an image is taken.Immediately following, the LED is turned on and a second image is taken.The difference between these images is used for spot detection. If nofog is present, the light will reflect from the windshield 26 at theSnell angle which will carry it well outside the field of view of theimage sensor 32. If fog is present, the light will be reflected byLambertian reflectance causing the fog to be imaged as a small spot 70(FIG. 3). Due to the angles of the windshield 26 and the light source66, outside fog will produce a spot 70 lower than a spot 72 produced byfog in the interior surface of the windshield 26. The location of thesespots 70, 72 may be used to represent the presence of interior and/orexterior fog on the windshield 26. If fog is present on the interior ofthe windshield, exterior fog detection is not possible. However, thislimitation is of no consequence because vision would be impaired anyway.The following truth table indicates the conclusions drawn from thepresence of each spot:

Exterior (Lower) Interior (Upper) Reflected Spot Reflected Spot ResultPresent — Exterior Fog — Present Interior Fog Not Present Not Present NoFog

In an alternate embodiment an infrared or, if necessary, a visible LEDis used. The LED must be either relatively small or be used inconjunction with a pin hole and aimed such that the light from the LEDreflects off the windshield 26 and on to an image sensor at the Snellangle. In such a configuration, two reflections will occur: one off theinside of the windshield and one from the outer surface of thewindshield by specular reflection. In this embodiment if fog is notpresent, the spots are reflected at the Snell angle and are visible bythe image sensor. If there is fog on the exterior surface of thewindshield 26, the spot from the interior reflection will be present butthe spot from the exterior refection will be a blur. The differenceimage is analyzed for interior fog. Initially a 3×3 Laplacian filter isused to create a new image which contains only the high frequencycomponents of the original image. In this way the blurred reflectionfrom fog is illuminated. The spots are detected by taking the maximumvalue of a group of pixels in the expected region of each spot. A groupslightly larger than the expected size of the spot may be used in orderto correct for slight misalignments and for the fact that the Laplacianwill only preserve the edges of the spots where the high frequencycomponents are present. Also slight fog may cause the spot to grow butmay not require action. For each spot, if the value is reasonablygreater than zero, it is determined to be present. The following truthtable indicates the detection of fog for the alternate embodiment.

Exterior (Upper) Interior (Lower) Reflected Spot Reflected Spot ResultPresent Present No Fog Not Present Present Exterior Fog Not Present NotPresent Interior Fog

If the exterior fog is detected, the windshield wipers may be actuatedto help remove the fog or if desired a warning light can be used toindicate this condition to the driver. Interior fog detection can beused to automatically actuate the vehicle defrost or defogger systempreventing the driver from waiting until the significant fog hasdeveloped. More involved processing methods may include determining thesize of the spot by computing the distance between the horizontal edgesof the spot and thus having a measurement of the quantity of fog on thewindshield which can be compared to a threshold.

Obviously, many modifications and variations of the present inventionare possible in light of the above teachings. Thus, it is to beunderstood that, within the scope of the appended claims, the inventionmay be practiced otherwise than as specifically described above.

What is claimed is:
 1. A windshield wiper control system for a vehiclecomprising: an image sensor; an optical system operative to image atleast a portion of the surface of the windshield onto the image sensor;a processing system in communication with the image sensor and operativeto analyze images from the image sensor to detect moisture on thewindshield and to activate the windshield wipers when moisture isdetected; and a user interface for allowing a user to adjust thesensitivity of said processing system to moisture when activating thewindshield wipers.
 2. The windshield wiper control system of claim 1,where said processing system determines a spatial frequency component ofthe moisture on the windshield and generates control signals to controlthe windshield wipers if the spatial frequency component is above athreshold, said user interface allowing a user to adjust the thresholdand thereby adjust the sensitivity of said processing system to moisturewhen activating the windshield wipers.
 3. The windshield wiper controlsystem of claim 1, where said processing system determines a spatialfrequency component of the moisture on the windshield and generatescontrol signals to control the time intervals at which the windshieldwipers are activated.
 4. The windshield wiper control system of claim 1,wherein said processing system analyzes images from the image sensor todetect the presence of a windshield wiper moving across the imagedportion of the surface.
 5. The windshield wiper control system of claim1, further comprising an illumination device for selectivelyilluminating an area within the at least one portion of the windshieldsurface, wherein said processing system analyzes images from said imagesensor to detect rain, snow, and debris on the windshield, saidprocessing system periodically activates said illumination device todetect fog on the inside of the windshield.
 6. A system for detectingmoisture on a surface comprising: an image sensor; an optical systemoperative to image at least a portion of the surface onto the imagesensor; an illumination device for selectively illuminating an areawithin the at least one portion of the windshield surface; and aprocessing system in communication with the image sensor and operativeto analyze images from the image sensor to detect rain, snow, and debrison the windshield, said processing system periodically activates saidillumination device to detect fog on the inside of the windshield. 7.The system of claim 6, further comprising a user interface for allowinga user to adjust the sensitivity of said processing system to moisturewhen activating the windshield wipers.
 8. The system of claim 6, whereinsaid processing system analyzes images from the image sensor to detectthe presence of a windshield wiper moving across the imaged portion ofthe surface.
 9. A system for detecting moisture on a surface of avehicle windshield comprising: an image sensor; an optical systemoperative to image at least a portion of the surface onto the imagesensor; and a processing system in communication with the image sensorand operative to analyze images from the image sensor to detect moistureand to detect the presence of a windshield wiper moving across theimaged portion of the surface.
 10. The system of claim 9, wherein saidprocessing system detects the presence of the windshield wiper bydetermining if a plurality of high frequency components in the directionof wiper travel through the imaged portion of the surface is greaterthan a plurality of high frequency components in a direction normal tothe direction of wiper travel through the imaged portion of the surface.11. The system of claim 9, further comprising a user interface forallowing a user to adjust the sensitivity of said processing system tomoisture when activating the windshield wipers.
 12. The system of claim9, further comprising an illumination device for selectivelyilluminating an area within the at least one portion of the windshieldsurface, wherein said processing system analyzes images from said imagesensor to detect rain, snow, and debris on the windshield, saidprocessing system periodically activates said illumination device todetect fog on the inside of the windshield.
 13. The system of claim 9,wherein, when moisture is detected and a windshield wiper is notdetected, said processing system generates a control signal to suspendoperation of the windshield wiper.
 14. The system of claim 13, whereinsaid processing system generates the control signal to suspend operationof the windshield wiper if said processing system does not detect thepresence of the wiper has within a predetermined time period.
 15. Thesystem of claim 9, wherein said processing system analyzes the imagesfrom said image sensor to detect moisture on the windshield immediatelyfollowing detection of the windshield wiper.
 16. The system of claim 15,wherein said processing system compares images from said image sensorwith those analyzed immediately following detection of the windshieldwiper to determine whether the images are different.
 17. The system ofclaim 16, wherein said processing system generates a control signal todeactivate the windshield wiper if said processing system determinesthat the compared images are not different.